Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Normalize RepoTags before checking for match #53161

Merged
merged 1 commit into from
Sep 28, 2017

Conversation

dims
Copy link
Member

@dims dims commented Sep 27, 2017

What this PR does / why we need it:

on projectatomic-based docker, we get "docker.io/library/busybox:latest"
when someone uses an unqualified name like "busybox". Though when we
inspect, the RepoTag will still say "docker.io/busybox:latest", So
we have reparse the tag, normalize it and try again. Please see the
additional test case.

Signed-off-by: Andy Goldstein andy.goldstein@gmail.com

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Fixes #52110

Special notes for your reviewer:

Release note:

Fixes an issue pulling pod specs referencing unqualified images from docker.io on centos/fedora/rhel

@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Sep 27, 2017
@k8s-github-robot k8s-github-robot added the do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. label Sep 27, 2017
@dims
Copy link
Member Author

dims commented Sep 27, 2017

/release-note-none
/sig node

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. sig/node Categorizes an issue or PR as relevant to SIG Node. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Sep 27, 2017
@@ -60,6 +60,13 @@ func matchImageTagOrSHA(inspected dockertypes.ImageInspect, image string) bool {
// hostname or not, we only check for the suffix match.
if strings.HasSuffix(image, tag) || strings.HasSuffix(tag, image) {
return true
} else {
t, _ := dockerref.ParseNormalizedNamed(tag)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this returns an error, we should continue, not ignore it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -60,6 +60,13 @@ func matchImageTagOrSHA(inspected dockertypes.ImageInspect, image string) bool {
// hostname or not, we only check for the suffix match.
if strings.HasSuffix(image, tag) || strings.HasSuffix(tag, image) {
return true
} else {
t, _ := dockerref.ParseNormalizedNamed(tag)
t2 := t.(dockerref.Tagged)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check the cast t2, ok := t.(dockerref.Tagged) and continue if !ok

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

} else {
t, _ := dockerref.ParseNormalizedNamed(tag)
t2 := t.(dockerref.Tagged)
normalizedTag := t2.String()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if normalizedTag is empty, continue

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@yujuhong
Copy link
Contributor

/cc @yguo0905 FYI

@k8s-ci-robot
Copy link
Contributor

@yujuhong: GitHub didn't allow me to request PR reviews from the following users: FYI.

Note that only kubernetes members can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @yguo0905 FYI

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@liggitt
Copy link
Member

liggitt commented Sep 27, 2017

this LGTM, manually building/verifying now

@liggitt
Copy link
Member

liggitt commented Sep 27, 2017

cc @miminar

@dims
Copy link
Member Author

dims commented Sep 27, 2017

Duh, needed to fix gofmt

@liggitt liggitt added this to the v1.8 milestone Sep 27, 2017
@liggitt liggitt added cherrypick-candidate kind/bug Categorizes issue or PR as related to a bug. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. labels Sep 27, 2017
@Random-Liu
Copy link
Member

Random-Liu commented Sep 27, 2017

@dims Please add comment to explain why we are doing this, and link to corresponding K8s issue. I'm worried that we may forget why we are doing this in the future.

And we should also fix docker on centos to properly return image reference so as to get rid of this hack in the future.

@dims
Copy link
Member Author

dims commented Sep 27, 2017

@Random-Liu where in the commit message itself?

@derekwaynecarr
Copy link
Member

I tested on Fedora 26

Installed Packages
docker-latest.x86_64                                                                               2:1.13-32.git27e468e.fc26          
$ docker info
Containers: 14
 Running: 6
 Paused: 0
 Stopped: 8
Images: 7
Server Version: 1.13.1
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: journald
Cgroup Driver: systemd
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: oci runc
Default Runtime: oci
Init Binary: /usr/libexec/docker/docker-init-latest
containerd version:  (expected: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1)
runc version: N/A (expected: 9df8b306d01f59d3a8029be411de015b7304dd8f)
init version: N/A (expected: 949e6facb77383876aeff8a6944dde66b3089574)
Security Options:
 seccomp
  WARNING: You're not using the default seccomp profile
  Profile: /etc/docker-latest/seccomp.json
 selinux
Kernel Version: 4.12.13-300.fc26.x86_64
Operating System: Fedora 26 (Workstation Edition)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 3
CPUs: 4
Total Memory: 19.48 GiB
Name: localhost.localdomain
ID: VP7V:3UZE:VKXX:UVWE:4M43:2MLI:E72F:4KIE:KC45:GEWG:RXRP:RCUX
Docker Root Dir: /var/lib/docker-latest
Debug Mode (client): false
Debug Mode (server): false
Username: derekwaynecarr
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Registries: docker.io (secure)

verified the following works as expected after this fix:

$ kubectl run -i -t busybox --image=busybox --restart=Never

@derekwaynecarr
Copy link
Member

/approve

@k8s-github-robot k8s-github-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 27, 2017
@dims
Copy link
Member Author

dims commented Sep 28, 2017

/test pull-kubernetes-e2e-gce-bazel

@dims
Copy link
Member Author

dims commented Sep 28, 2017

/test pull-kubernetes-unit

@@ -60,6 +60,38 @@ func matchImageTagOrSHA(inspected dockertypes.ImageInspect, image string) bool {
// hostname or not, we only check for the suffix match.
if strings.HasSuffix(image, tag) || strings.HasSuffix(tag, image) {
return true
} else {
// TODO: We need to remove this hack when project atomic based
// docker distro(s) like centos/fedora/rhel image fix problems on
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

double space.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks fixed.

@Random-Liu
Copy link
Member

LGTM with one nit.

on projectatomic-based docker, we get "docker.io/library/busybox:latest"
when someone uses an unqualified name like "busybox". Though when we
inspect, the RepoTag will still say "docker.io/busybox:latest", So
we have reparse the tag, normalize it and try again. Please see the
additional test case.

Signed-off-by: Andy Goldstein <andy.goldstein@gmail.com>
@ncdc
Copy link
Member

ncdc commented Sep 28, 2017

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 28, 2017
@k8s-github-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: derekwaynecarr, dims, ncdc

Associated issue: 52110

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these OWNERS Files:

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

@ncdc
Copy link
Member

ncdc commented Sep 28, 2017

@dims thanks for carrying this forward

@dims
Copy link
Member Author

dims commented Sep 28, 2017

/test pull-kubernetes-e2e-gce-bazel

@liggitt liggitt added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Sep 28, 2017
@sjenning
Copy link
Contributor

verified fix CentOS 7.4.1708

# cat /etc/centos-release
CentOS Linux release 7.4.1708 (Core)
# docker version
Client:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-1.12.6-55.gitc4618fb.el7.centos.x86_64
 Go version:      go1.8.3
 Git commit:      c4618fb/1.12.6
 Built:           Thu Sep 21 22:33:52 2017
 OS/Arch:         linux/amd64

Server:
 Version:         1.12.6
 API version:     1.24
 Package version: docker-1.12.6-55.gitc4618fb.el7.centos.x86_64
 Go version:      go1.8.3
 Git commit:      c4618fb/1.12.6
 Built:           Thu Sep 21 22:33:52 2017
 OS/Arch:         linux/amd64

@k8s-github-robot
Copy link

Automatic merge from submit-queue (batch tested with PRs 52634, 53121, 53161). If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-github-robot k8s-github-robot merged commit 85c37d7 into kubernetes:master Sep 28, 2017
k8s-github-robot pushed a commit that referenced this pull request Sep 28, 2017
…1-upstream-release-1.8

Automatic merge from submit-queue.

Automated cherry pick of #53161

Cherry pick of #53161 on release-1.8.

#53161: Normalize RepoTags before checking for match

```release-note
Fixes an issue pulling pod specs referencing unqualified images from docker.io on centos/fedora/rhel
```
@k8s-cherrypick-bot
Copy link

Commit found in the "release-1.8" branch appears to be this PR. Removing the "cherrypick-candidate" label. If this is an error find help to get your PR picked.

@dims dims deleted the fix-repotags branch November 16, 2017 22:07
openshift-merge-robot added a commit to openshift/origin that referenced this pull request Jan 31, 2018
Automatic merge from submit-queue (batch tested with PRs 18225, 18351, 18331, 18340, 18326).

UPSTREAM: 58955: pkg: kubelet: do not assume anything about images names

kubernetes/kubernetes#58955

removes the need for #18020

see history:
kubernetes/kubernetes#51751
kubernetes/kubernetes#52110
kubernetes/kubernetes#53161

@liggitt @derekwaynecarr @frobware @mrunalp @runcom
openshift-publish-robot pushed a commit to openshift/kubernetes that referenced this pull request Jan 31, 2018
Automatic merge from submit-queue (batch tested with PRs 18225, 18351, 18331, 18340, 18326).

UPSTREAM: 58955: pkg: kubelet: do not assume anything about images names

kubernetes#58955

removes the need for openshift/origin#18020

see history:
kubernetes#51751
kubernetes#52110
kubernetes#53161

@liggitt @derekwaynecarr @frobware @mrunalp @runcom

Origin-commit: d91de347457d7f6f3d1859c671c7355cc193d038
openshift-publish-robot pushed a commit to openshift/kubernetes that referenced this pull request Feb 27, 2018
Automatic merge from submit-queue (batch tested with PRs 18225, 18351, 18331, 18340, 18326).

UPSTREAM: 58955: pkg: kubelet: do not assume anything about images names

kubernetes#58955

removes the need for openshift/origin#18020

see history:
kubernetes#51751
kubernetes#52110
kubernetes#53161

@liggitt @derekwaynecarr @frobware @mrunalp @runcom

Origin-commit: d91de347457d7f6f3d1859c671c7355cc193d038
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/node Categorizes an issue or PR as relevant to SIG Node. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1.8 regression: Can't pull images from docker.io without explicit path on centos/fedora/rhel